Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Chore] Reorganize engine.ts #536

Merged
merged 1 commit into from
Aug 10, 2024
Merged

[Chore] Reorganize engine.ts #536

merged 1 commit into from
Aug 10, 2024

Conversation

CharlieFRuan
Copy link
Contributor

@CharlieFRuan CharlieFRuan commented Aug 10, 2024 •
edited
Loading

This PR reorganizes engine.ts to keep it relatively more organized.

  • We move getToolCallFromOutputMessage() from engine.ts to support.ts since it does not depend on any engine-related fields and is stateless
  • We move asyncLoadTokenizer() from engine.ts to cache_util.ts for the same reason
  • In future, we can implement a common helper for chatCompletion() and completion()
  • For engine.ts, we break down the functions into sections:
  1. Setters and getters
  2. Model/pipeline loading and unloading: reload(), unload()
  3. Underlying auto-regressive generation functions: _generate(), asyncGenerate(), interruptGenerate()
  4. High-level generation APIs: generate() (deprecated), chatCompletion(), completion()
  5. WebGPU info-querying helpers. Arguably, this can be implemented in support.ts as well. Currently these methods are in the MLCEngineInterface, and goes through the WebWorkerMLCEngine - MLCEngine message passing workflow
  6. Low-level APIs that interact with pipeline

@CharlieFRuan CharlieFRuan merged commit d21f00e into mlc-ai:main Aug 10, 2024
1 check passed
@CharlieFRuan CharlieFRuan mentioned this pull request Aug 12, 2024
Merged
CharlieFRuan added a commit that referenced this pull request Aug 12, 2024
@CharlieFRuan
[Version][Trivial] Bump version to 0.2.57 (#537)
66dd646 
### Changes
- Reorganized internal code, which should have no affect on behaviors
#536
- Bumped version of tokenizers-cpp to fix potential error `The global
thread pool has not been initialized.`
  - For more, see mlc-ai/tokenizers-cpp#42

### TVMjs
- Still compiled at
apache/tvm@1fcb620,
no change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@CharlieFRuan